The present invention relates generally to the field of regression testing, and more particularly to proactive and selective regression testing based on historic test results.
Regression testing is a manual development process in which a code developer creates test cases upon completion of a code change and executes the test function to determine if the new code functions properly and does not give rise to unwanted side effects arising from interaction between added features and existing code. Generally, in a development and verification environment, a large set of regression test cases and/or benchmarks is generated and retained for analysis of design revisions and extensions. In the verification environment in which a large set of regression test cases exists, the testing of a new version of the design may require a significant amount of time in order to perform each and every one of the regression test cases. Often, reasonable balance is sought to avoid testing every possible available regression test case, while still achieving reasonable confidence in software quality.
Large software products, such as compilers, have complex and sensitive code, where a change in one component can uncover bad functionality that has gone undetected in another component. Some software products also have large, resource-intensive bodies of tests, leading to functional testing that can comprise tens of thousands of test cases and take several days to execute. In addition, a single performance benchmark can take over a day to compile. Thus testing may only be performed on a weekly basis instead of after each code change.